Tootfinder

This https://arxiv.org/abs/2311.12871 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCV_…

An Embodied Generalist Agent in 3D World
Leveraging massive knowledge from large language models (LLMs), recent machine learning models show notable successes in general-purpose task solving in diverse domains such as computer vision and robotics. However, several significant challenges remain: (i) most of these models rely on 2D images yet exhibit a limited capacity for 3D input; (ii) these models rarely explore the tasks inherently defined in 3D world, e.g., 3D grounding, embodied reasoning and acting. We argue these limitations sig…

Probing Multimodal LLMs as World Models for Driving
Shiva Sreeram, Tsun-Hsuan Wang, Alaa Maalouf, Guy Rosman, Sertac Karaman, Daniela Rus
https://arxiv.org/abs/2405.05956

Probing Multimodal LLMs as World Models for Driving
We provide a sober look at the application of Multimodal Large Language Models (MLLMs) within the domain of autonomous driving and challenge/verify some common assumptions, focusing on their ability to reason and interpret dynamic driving scenarios through sequences of images/frames in a closed-loop control environment. Despite the significant advancements in MLLMs like GPT-4V, their performance in complex, dynamic driving environments remains largely untested and presents a wide area of explor…

CausalBench: A Comprehensive Benchmark for Causal Learning Capability of Large Language Models
Yu Zhou, Xingyu Wu, Beicheng Huang, Jibin Wu, Liang Feng, Kay Chen Tan
https://arxiv.org/abs/2404.06349

CausalBench: A Comprehensive Benchmark for Causal Learning Capability of Large Language Models
Causality reveals fundamental principles behind data distributions in real-world scenarios, and the capability of large language models (LLMs) to understand causality directly impacts their efficacy across explaining outputs, adapting to new evidence, and generating counterfactuals. With the proliferation of LLMs, the evaluation of this capacity is increasingly garnering attention. However, the absence of a comprehensive benchmark has rendered existing evaluation studies being straightforward, …

This https://arxiv.org/abs/2312.09979 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks. Increasing instruction data substantially is a direct solution to align the model with a broader range of downstream tasks or notably improve its performance on a specific task. However, we find that large-scale increases in instruction data can damage the world knowledge previously stored in LLMs. To address this cha…

Sometimes i forget there is more then the bigtech players in generative AI. A good example is #AI

Cohere | The leading AI platform for enterprise
Cohere provides industry-leading large language models (LLMs) and RAG capabilities tailored to meet the needs of enterprise use cases that solve real-world problems.

Responsible Generative AI: What to Generate and What Not
Jindong Gu
https://arxiv.org/abs/2404.05783 https://arxiv.org/pdf/2404.05783…

Responsible Generative AI: What to Generate and What Not
In recent years, generative AI (GenAI), like large language models and text-to-image models, has received significant attention across various domains. However, ensuring the responsible generation of content by these models is crucial for their real-world applicability. This raises an interesting question: \textit{What should responsible GenAI generate, and what should it not?} To answer the question, this paper investigates the practical responsible requirements of both textual and visual gene…

Cost of Locally Approximating High-Dimensional Ground States of Contextual Quantum Models
Kaiyan Yang, Yanzheng Zhu, Xiao Zeng, Zuoheng Zou, Man-Hong Yung, Zizhu Wang
https://arxiv.org/abs/2405.04884

Cost of Locally Approximating High-Dimensional Ground States of Contextual Quantum Models
Contextuality, one of the strongest forms of quantum correlations, delineates the quantum world and the classical one. It has been shown recently that some quantum models, in the form of infinite one-dimensional translation-invariant Hamiltonians with nearest- and next-to-nearest-neighbor interactions, have the lowest ground state energy density allowed in quantum physics. However, these models all have local Hilbert space dimension larger than two, making the study of their ground state behavi…

Challenges Faced by Large Language Models in Solving Multi-Agent Flocking
Peihan Li, Vishnu Menon, Bhavanaraj Gudiguntla, Daniel Ting, Lifeng Zhou
https://arxiv.org/abs/2404.04752

Challenges Faced by Large Language Models in Solving Multi-Agent Flocking
Flocking is a behavior where multiple agents in a system attempt to stay close to each other while avoiding collision and maintaining a desired formation. This is observed in the natural world and has applications in robotics, including natural disaster search and rescue, wild animal tracking, and perimeter surveillance and patrol. Recently, large language models (LLMs) have displayed an impressive ability to solve various collaboration tasks as individual decision-makers. Solving multi-agent f…

This https://arxiv.org/abs/2405.03690 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCV_…

Complex Video Reasoning and Robustness Evaluation Suite for Video-LMMs
Recent advancements in Large Language Models (LLMs) have led to the development of Video Large Multi-modal Models (Video-LMMs) that can handle a wide range of video understanding tasks. These models have the potential to be deployed in real-world applications such as robotics, AI assistants, medical imaging, and autonomous vehicles. The widespread adoption of Video-LMMs in our daily lives underscores the importance of ensuring and evaluating their robust performance in mirroring human-like reas…

Artificial intelligence for abnormality detection in high volume neuroimaging: a systematic review and meta-analysis
Siddharth Agarwal, David A. Wood, Mariusz Grzeda, Chandhini Suresh, Munaib Din, James Cole, Marc Modat, Thomas C Booth
https://arxiv.org/abs/2405.05658

Artificial intelligence for abnormality detection in high volume neuroimaging: a systematic review and meta-analysis
Purpose: Most studies evaluating artificial intelligence (AI) models that detect abnormalities in neuroimaging are either tested on unrepresentative patient cohorts or are insufficiently well-validated, leading to poor generalisability to real-world tasks. The aim was to determine the diagnostic test accuracy and summarise the evidence supporting the use of AI models performing first-line, high-volume neuroimaging tasks. Methods: Medline, Embase, Cochrane library and Web of Science were searc…

LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements
Victoria Basmov, Yoav Goldberg, Reut Tsarfaty
https://arxiv.org/abs/2404.06283

LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements
The task of reading comprehension (RC), often implemented as context-based question answering (QA), provides a primary means to assess language models' natural language understanding (NLU) capabilities. Yet, when applied to large language models (LLMs) with extensive built-in world knowledge, this method can be deceptive. If the context aligns with the LLMs' internal knowledge, it is hard to discern whether the models' answers stem from context comprehension or from LLMs' internal information. …

Temporal True and Surrogate Fitness Landscape Analysis for Expensive Bi-Objective Optimisation
C. J. Rodriguez, S. L. Thomson, T. Alderliesten, P. A. N. Bosman
https://arxiv.org/abs/2404.06557

Temporal True and Surrogate Fitness Landscape Analysis for Expensive Bi-Objective Optimisation
Many real-world problems have expensive-to-compute fitness functions and are multi-objective in nature. Surrogate-assisted evolutionary algorithms are often used to tackle such problems. Despite this, literature about analysing the fitness landscapes induced by surrogate models is limited, and even non-existent for multi-objective problems. This study addresses this critical gap by comparing landscapes of the true fitness function with those of surrogate models for multi-objective functions. Mo…

Vision-Language Model-based Physical Reasoning for Robot Liquid Perception
Wenqiang Lai, Yuan Gao, Tin Lun Lam
https://arxiv.org/abs/2404.06904 https://

Vision-Language Model-based Physical Reasoning for Robot Liquid Perception
There is a growing interest in applying large language models (LLMs) in robotic tasks, due to their remarkable reasoning ability and extensive knowledge learned from vast training corpora. Grounding LLMs in the physical world remains an open challenge as they can only process textual input. Recent advancements in large vision-language models (LVLMs) have enabled a more comprehensive understanding of the physical world by incorporating visual input, which provides richer contextual information t…

Policy-Guided Diffusion
Matthew Thomas Jackson, Michael Tryfan Matthews, Cong Lu, Benjamin Ellis, Shimon Whiteson, Jakob Foerster
https://arxiv.org/abs/2404.06356

Policy-Guided Diffusion
In many real-world settings, agents must learn from an offline dataset gathered by some prior behavior policy. Such a setting naturally leads to distribution shift between the behavior policy and the target policy being trained - requiring policy conservatism to avoid instability and overestimation bias. Autoregressive world models offer a different solution to this by generating synthetic, on-policy experience. However, in practice, model rollouts must be severely truncated to avoid compoundin…

If copyright is real then the AI machines owe the artists of humankind (which is to say all of us) more money than exists in the banking systems of the planet.
If copyright is not real then the companies that trained those models have no right to exclusive ownership of the weights.
I bet somehow the law conspires to make it the worst of either option.
#ai #copyright #copyleft

CodeFort: Robust Training for Code Generation Models
Yuhao Zhang, Shiqi Wang, Haifeng Qian, Zijian Wang, Mingyue Shang, Linbo Liu, Sanjay Krishna Gouda, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras
https://arxiv.org/abs/2405.01567

CodeFort: Robust Training for Code Generation Models
Code generation models are not robust to small perturbations, which often lead to inconsistent and incorrect generations and significantly degrade the performance of these models. Improving the robustness of code generation models is crucial to better user experience when these models are deployed in real-world applications. However, existing efforts have not addressed this issue for code generation models. To fill this gap, we propose CodeFort, a framework to improve the robustness of code gen…

Defending Against Unforeseen Failure Modes with Latent Adversarial Training
Stephen Casper, Lennart Schulze, Oam Patel, Dylan Hadfield-Menell
https://arxiv.org/abs/2403.05030

Defending Against Unforeseen Failure Modes with Latent Adversarial Training
AI systems sometimes exhibit harmful unintended behaviors post-deployment. This is often despite extensive diagnostics and debugging by developers. Minimizing risks from models is challenging because the attack surface is so large. It is not tractable to exhaustively search for inputs that may cause a model to fail. Red-teaming and adversarial training (AT) are commonly used to make AI systems more robust. However, they have not been sufficient to avoid many real-world failure modes that differ…

Cave art
Stone tablets
Papyrus scrolls
Paper, pencil, pen, and ink
Printing presses
Telegraphs
Typewriters
Computers
Internet
Tablets
Social media
Large language models
Large world models
Cave art

#KI ziert jede Sonntagsrede, aber woher kommen eigentlich die Menschen, die das alles aufbauen und wo gehen sie hin? Das zeigt die aktuelle Fassung des Global AI Talent Tracker:
https://macropolo.…

VISION2UI: A Real-World Dataset with Layout for Code Generation from UI Designs
Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Yi Su, Shaoling Dong, Xing Zhou, Wenbin Jiang
https://arxiv.org/abs/2404.06369

VISION2UI: A Real-World Dataset with Layout for Code Generation from UI Designs
Automatically generating UI code from webpage design visions can significantly alleviate the burden of developers, enabling beginner developers or designers to directly generate Web pages from design diagrams. Currently, prior research has accomplished the objective of generating UI code from rudimentary design visions or sketches through designing deep neural networks. Inspired by the groundbreaking advancements achieved by Multimodal Large Language Models (MLLMs), the automatic generation of …

This https://arxiv.org/abs/2310.09499 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
Various Large Language Models(LLMs) from the Generative Pretrained Transformer(GPT) family have achieved outstanding performances in a wide range of text generation tasks. However, the enormous model sizes have hindered their practical use in real-world applications due to high inference latency. Therefore, improving the efficiencies of LLMs through quantization, pruning, and other means has been a key issue in LLM studies. In this work, we propose a method based on Hessian sensitivity-aware mi…

This https://arxiv.org/abs/2402.10100 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSD_…

Tuning In: Analysis of Audio Classifier Performance in Clinical Settings with Limited Data
This study assesses deep learning models for audio classification in a clinical setting with the constraint of small datasets reflecting real-world prospective data collection. We analyze CNNs, including DenseNet and ConvNeXt, alongside transformer models like ViT, SWIN, and AST, and compare them against pre-trained audio models such as YAMNet and VGGish. Our method highlights the benefits of pre-training on large datasets before fine-tuning on specific clinical data. We prospectively collected…

Very important:
"Investigating #trainingsets is an essential avenue to understanding how #generativeAI models work; the ways they see and re-create the world."

In reference to this: https://toot.cafe/@baldur/112031199817039932
I think many people in tech live and have lived a sheltered life that never required them to do anything outside of it (and their comfort zone). This does have massive impact on how they see the world, whose skills they t…

World's
Tiniest
Violin
OpenAI accuses New York Times of hacking AI models in copyright lawsuit
https://cointelegraph.com/news/openai-new-york-times-hacking-ai-models

Archetype, which is building AI models to help humans understand the data from sensors monitoring the physical world, launches with a $13M seed led by Venrock (Steven Levy/Wired)
https://www.wired.com/story/plaintext-ai-startup-archety…

I wonder how many people who, at every turn, are breathlessly shilling/hawking corporate-owned 'AI' (a marketing term that really just means large language models or a 'souped up predictive text' service) really think they're making the world better with their efforts...

This https://arxiv.org/abs/2310.04003 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csNI_…

The Role of Federated Learning in a Wireless World with Foundation Models
Foundation models (FMs) are general-purpose artificial intelligence (AI) models that have recently enabled multiple brand-new generative AI applications. The rapid advances in FMs serve as an important contextual backdrop for the vision of next-generation wireless networks, where federated learning (FL) is a key enabler of distributed network intelligence. Currently, the exploration of the interplay between FMs and FL is still in its nascent stage. Naturally, FMs are capable of boosting the per…

Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model
Zhonghan Zhao, Ke Ma, Wenhao Chai, Xuan Wang, Kewei Chen, Dongxu Guo, Yanting Zhang, Hongwei Wang, Gaoang Wang
https://arxiv.org/abs/2404.04619

Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model
With the power of large language models (LLMs), open-ended embodied agents can flexibly understand human instructions, generate interpretable guidance strategies, and output executable actions. Nowadays, Multi-modal Language Models~(MLMs) integrate multi-modal signals into LLMs, further bringing richer perception to entity agents and allowing embodied agents to perceive world-understanding tasks more delicately. However, existing works: 1) operate independently by agents, each containing multip…

The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models
Abeba Birhane, Sepehr Dehdashtian, Vinay Uday Prabhu, Vishnu Boddeti
https://arxiv.org/abs/2405.04623

The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models
Scale the model, scale the data, scale the GPU farms is the reigning sentiment in the world of generative AI today. While model scaling has been extensively studied, data scaling and its downstream impacts on model performance remain under-explored. This is particularly important in the context of multimodal datasets whose main source is the World Wide Web, condensed and packaged as the Common Crawl dump, which is known to exhibit numerous drawbacks. In this paper, we evaluate the downstream im…

This https://arxiv.org/abs/2107.02739 has been replaced.
link: https://scholar.google.com/scholar?q=a

Shapes as Product Differentiation: Neural Network Embedding in the Analysis of Markets for Fonts
Many differentiated products have key attributes that are unstructured and thus high-dimensional (e.g., design, text). Instead of treating unstructured attributes as unobservables in economic models, quantifying them can be important to answer interesting economic questions. To propose an analytical framework for these types of products, this paper considers one of the simplest design products-fonts-and investigates merger and product differentiation using an original dataset from the world's l…

Teacher-Student Network for Real-World Face Super-Resolution with Progressive Embedding of Edge Information
Zhilei Liu, Chenggong Zhang
https://arxiv.org/abs/2405.04778

Teacher-Student Network for Real-World Face Super-Resolution with Progressive Embedding of Edge Information
Traditional face super-resolution (FSR) methods trained on synthetic datasets usually have poor generalization ability for real-world face images. Recent work has utilized complex degradation models or training networks to simulate the real degradation process, but this limits the performance of these methods due to the domain differences that still exist between the generated low-resolution images and the real low-resolution images. Moreover, because of the existence of a domain gap, the seman…

Defending Against Unforeseen Failure Modes with Latent Adversarial Training
Stephen Casper, Lennart Schulze, Oam Patel, Dylan Hadfield-Menell
https://arxiv.org/abs/2403.05030

Defending Against Unforeseen Failure Modes with Latent Adversarial Training
AI systems sometimes exhibit harmful unintended behaviors post-deployment. This is often despite extensive diagnostics and debugging by developers. Minimizing risks from models is challenging because the attack surface is so large. It is not tractable to exhaustively search for inputs that may cause a model to fail. Red-teaming and adversarial training (AT) are commonly used to make AI systems more robust. However, they have not been sufficient to avoid many real-world failure modes that differ…

Towards Efficient and Effective Unlearning of Large Language Models for Recommendation
Hangyu Wang, Jianghao Lin, Bo Chen, Yang Yang, Ruiming Tang, Weinan Zhang, Yong Yu
https://arxiv.org/abs/2403.03536

Towards Efficient and Effective Unlearning of Large Language Models for Recommendation
The significant advancements in large language models (LLMs) give rise to a promising research direction, i.e., leveraging LLMs as recommenders (LLMRec). The efficacy of LLMRec arises from the open-world knowledge and reasoning capabilities inherent in LLMs. LLMRec acquires the recommendation capabilities through instruction tuning based on user interaction data. However, in order to protect user privacy and optimize utility, it is also crucial for LLMRec to intentionally forget specific user d…

Eine Studie der Wageningen Universität zeigt, warum der #Nordpol schneller erwärmt wird als erwartet. Es wurden Daten von einer #Forschungsreise im Arktischen Ozean genutzt. Demnach tragen Ozon und warme Luftströme wesentlich zur Erwärmung bei. Ozon wirkt als

Uncovering the reasons behind the rapid warming of the North Pole
The North Pole region heats up faster than the rest of the world. Though this is a known fact, climate models underestimate the speed with which the region warms up. Sjoert Barten obtained his PhD on this subject at Wageningen University & Research on 26 April and shares his insights.

This https://arxiv.org/abs/2404.03543 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
Large Language Models (LLMs) for code are rapidly evolving, with code editing emerging as a critical capability. We introduce CodeEditorBench, an evaluation framework designed to rigorously assess the performance of LLMs in code editing tasks, including debugging, translating, polishing, and requirement switching. Unlike existing benchmarks focusing solely on code generation, CodeEditorBench emphasizes real-world scenarios and practical aspects of software development. We curate diverse coding …

FlockGPT: Guiding UAV Flocking with Linguistic Orchestration
Artem Lykov, Sausar Karaf, Mikhail Martynov, Valerii Serpiva, Aleksey Fedoseev, Mikhail Konenkov, Dzmitry Tsetserukou
https://arxiv.org/abs/2405.05872

FlockGPT: Guiding UAV Flocking with Linguistic Orchestration
This article presents the world's first rapid drone flocking control using natural language through generative AI. The described approach enables the intuitive orchestration of a flock of any size to achieve the desired geometry. The key feature of the method is the development of a new interface based on Large Language Models to communicate with the user and to generate the target geometry descriptions. Users can interactively modify or provide comments during the construction of the flock geo…

«Models All The Way Down» is a really interesting exploration of how an image data set (in this case LAION-5B) was assembled to be used for training ML/"AI" models!
> "It contains less about how humans see the world than it does about how search engines see the world. It is a dataset that is powerfully shaped by commercial logics."
https://knowingmachines.org/models-all-the-way

This https://arxiv.org/abs/2309.02050 has been replaced.
link: https://scholar.google.com/scholar?q=a

Machine learning of network inference enhancement from noisy measurements
Inferring networks from observed time series data presents a clear glimpse into the interconnections among nodes. Network inference models, when dealing with real-world open cases, especially in the presence of observational noise, experience a sharp decline in performance, significantly undermining their practical applicability. We find that in real-world scenarios, noisy samples cause parameter updates in network inference models to deviate from the correct direction, leading to a degradation…

Ancestor regression in structural vector autoregressive models
Christoph Schultheiss, Peter B\"uhlmann
https://arxiv.org/abs/2403.03778 https://

Ancestor regression in structural vector autoregressive models
We present a new method for causal discovery in linear structural vector autoregressive models. We adapt an idea designed for independent observations to the case of time series while retaining its favorable properties, i.e., explicit error control for false causal discovery, at least asymptotically. We apply our method to several real-world bivariate time series datasets and discuss its findings which mostly agree with common understanding. The arrow of time in a model can be interpreted as ba…

BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language Models
Chu Fei Luo, Ahmad Ghawanmeh, Xiaodan Zhu, Faiza Khan Khattak
https://arxiv.org/abs/2405.04756

BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language Models
Modern large language models (LLMs) have a significant amount of world knowledge, which enables strong performance in commonsense reasoning and knowledge-intensive tasks when harnessed properly. The language model can also learn social biases, which has a significant potential for societal harm. There have been many mitigation strategies proposed for LLM safety, but it is unclear how effective they are for eliminating social biases. In this work, we propose a new methodology for attacking langu…

This https://arxiv.org/abs/2403.01644 has been replaced.
link: https://scholar.google.com/scholar?q=a

OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Prediction
A comprehensive understanding of 3D scenes is crucial in autonomous vehicles (AVs), and recent models for 3D semantic occupancy prediction have successfully addressed the challenge of describing real-world objects with varied shapes and classes. However, existing methods for 3D occupancy prediction heavily rely on surround-view camera images, making them susceptible to changes in lighting and weather conditions. This paper introduces OccFusion, a novel sensor fusion framework for predicting 3D …

Certain and Approximately Certain Models for Statistical Learning
Cheng Zhen, Nischal Aryal, Arash Termehchy, Amandeep Singh Chabada
https://arxiv.org/abs/2402.17926

Certain and Approximately Certain Models for Statistical Learning
Real-world data is often incomplete and contains missing values. To train accurate models over real-world datasets, users need to spend a substantial amount of time and resources imputing and finding proper values for missing data items. In this paper, we demonstrate that it is possible to learn accurate models directly from data with missing values for certain training data and target models. We propose a unified approach for checking the necessity of data imputation to learn accurate models a…

This https://arxiv.org/abs/2404.13236 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

LLMChain: Blockchain-based Reputation System for Sharing and Evaluating Large Language Models
Large Language Models (LLMs) have witnessed rapid growth in emerging challenges and capabilities of language understanding, generation, and reasoning. Despite their remarkable performance in natural language processing-based applications, LLMs are susceptible to undesirable and erratic behaviors, including hallucinations, unreliable reasoning, and the generation of harmful content. These flawed behaviors undermine trust in LLMs and pose significant hurdles to their adoption in real-world applic…

This https://arxiv.org/abs/2405.01358 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Propagation measurements and channel models in Indoor Environment at 6.75 GHz FR1(C) and 16.95 GHz FR3 Upper-mid band Spectrum for 5G and 6G
New spectrum allocations in the 4--8 GHz FR1(C) and 7--24 GHz FR3 mid-band frequency spectrum are being considered for 5G/6G cellular deployments. This paper presents results from the world's first comprehensive indoor hotspot (InH) propagation measurement campaign at 6.75 GHz and 16.95 GHz in the NYU WIRELESS Research Center using a 1 GHz wideband channel sounder system over distances from 11 to 97 m in line-of-sight (LOS) and non-LOS (NLOS). Analysis of directional and omnidirectional path lo…

Ofcourse results needs to be verified and confirmed in practice but after reading the
MedGemini paper from Google there is no doubt in my mind AI will change the world of medicines. Not replacing people but augmenting them during diagnosis, operations and treatment of patients.
https://arxiv.org/abs/2404.18416

Capabilities of Gemini Models in Medicine
Excellence in a wide variety of medical applications poses considerable challenges for AI, requiring advanced reasoning, access to up-to-date medical knowledge and understanding of complex multimodal data. Gemini models, with strong general capabilities in multimodal and long-context reasoning, offer exciting possibilities in medicine. Building on these core strengths of Gemini, we introduce Med-Gemini, a family of highly capable multimodal models that are specialized in medicine with the abili…

World Models for Autonomous Driving: An Initial Survey
Yanchen Guan, Haicheng Liao, Zhenning Li, Guohui Zhang, Chengzhong Xu
https://arxiv.org/abs/2403.02622

World Models for Autonomous Driving: An Initial Survey
In the rapidly evolving landscape of autonomous driving, the capability to accurately predict future events and assess their implications is paramount for both safety and efficiency, critically aiding the decision-making process. World models have emerged as a transformative approach, enabling autonomous driving systems to synthesize and interpret vast amounts of sensor data, thereby predicting potential future scenarios and compensating for information gaps. This paper provides an initial revi…

A great piece from @… - categories, models, and cognitive traps.
"We... experience the world in simplified snapshots, crammed into generalized, abstracted categories captured by language and represented by symbols.
It helps us survive, navigating an infinitely complex world. But now, we... have gone too far in the other direction—paying too muc…

Funes the Memorious and the Useful Lie of Categories
We've evolved to misunderstand our world. Models are making it worse. Here's how to think smarter, by channeling our inner Funes the Memorious.

BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language Models
Chu Fei Luo, Ahmad Ghawanmeh, Xiaodan Zhu, Faiza Khan Khattak
https://arxiv.org/abs/2405.04756

BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language Models
Modern large language models (LLMs) have a significant amount of world knowledge, which enables strong performance in commonsense reasoning and knowledge-intensive tasks when harnessed properly. The language model can also learn social biases, which has a significant potential for societal harm. There have been many mitigation strategies proposed for LLM safety, but it is unclear how effective they are for eliminating social biases. In this work, we propose a new methodology for attacking langu…

I regret to inform you that AI beauty pageants are a thing now.
A company called Fanvue, which is a subscription-based content creator platform along the same lines as OnlyFans,
recently teamed up with the World AI Creator Awards (WAICA)
to launch the world’s first “Miss AI” competition.
A team of judges – comprising two humans and two virtual models – will sort through AI-generated pictures of women and choose one to crown as “Miss AI”.
The winner gets a cas…

This https://arxiv.org/abs/2310.04003 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csNI_…

The Role of Federated Learning in a Wireless World with Foundation Models
Foundation models (FMs) are general-purpose artificial intelligence (AI) models that have recently enabled multiple brand-new generative AI applications. The rapid advances in FMs serve as an important contextual backdrop for the vision of next-generation wireless networks, where federated learning (FL) is a key enabler of distributed network intelligence. Currently, the exploration of the interplay between FMs and FL is still in its nascent stage. Naturally, FMs are capable of boosting the per…

CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding
Kaiyuan Chen, Xingzhuo Guo, Yu Zhang, Jianmin Wang, Mingsheng Long
https://arxiv.org/abs/2405.02384

CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding
Predictive Coding (PC) is a theoretical framework in cognitive science suggesting that the human brain processes cognition through spatiotemporal prediction of the visual world. Existing studies have developed spatiotemporal prediction neural networks based on the PC theory, emulating its two core mechanisms: Correcting predictions from residuals and hierarchical learning. However, these models do not show the enhancement of prediction skills on real-world forecasting tasks and ignore the Preci…

PRobELM: Plausibility Ranking Evaluation for Language Models
Zhangdie Yuan, Chenxi Whitehouse, Eric Chamoun, Rami Aly, Andreas Vlachos
https://arxiv.org/abs/2404.03818

PRobELM: Plausibility Ranking Evaluation for Language Models
This paper introduces PRobELM (Plausibility Ranking Evaluation for Language Models), a benchmark designed to assess language models' ability to discern more plausible from less plausible scenarios through their parametric knowledge. While benchmarks such as TruthfulQA emphasise factual accuracy or truthfulness, and others such as COPA explore plausible scenarios without explicitly incorporating world knowledge, PRobELM seeks to bridge this gap by evaluating models' capabilities to prioritise pl…

http://tid.bl.it/your-80211n-airport-network
In the 234-page "Take Control of Your 802.11n AirPort Network, Second Edition," Glenn provides real-world advice for configuring the 802.11n models of Apple's AirPort Express, AirPort Extreme, and Time Capsule.

This https://arxiv.org/abs/2402.06326 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csAI_…

Prompt Learning on Temporal Interaction Graphs
Temporal Interaction Graphs (TIGs) are widely utilized to represent real-world systems. To facilitate representation learning on TIGs, researchers have proposed a series of TIG models. However, these models are still facing two tough gaps between the pre-training and downstream predictions in their ``pre-train, predict'' training paradigm. First, the temporal discrepancy between the pre-training and inference data severely undermines the models' applicability in distant future predictions on th…

Hip new campaign: "The Millennium Tower - Lean Into It"

Entering the #fediverse
"…some of the alternatives that people went to after the collapse of Twitter are these newer models where you're not investing into a #platform as such, you're investing into a handle that you can kind of take with you.
So if you spend time cultivating, say,…

Social media is more 'chaotic' and 'fragmented' than ever, so what's the future of how we live online?
If you've lived online long enough for Facebook to start dredging up embarrassing status updates from a decade ago, chances are you've picked up on a distinct vibe shift in our virtual world.

Contextual API Completion for Unseen Repositories Using LLMs
Noor Nashid, Taha Shabani, Parsa Alian, Ali Mesbah
https://arxiv.org/abs/2405.04600 https://…

Contextual API Completion for Unseen Repositories Using LLMs
Large language models have made substantial progress in addressing diverse code-related tasks. However, their adoption is hindered by inconsistencies in generating output due to the lack of real-world, domain-specific information, such as for intra-repository API calls for unseen software projects. We introduce a novel technique to mitigate hallucinations by leveraging global and local contextual information within a code repository for API completion tasks. Our approach is tailored to refine c…

This https://arxiv.org/abs/2405.01649 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
Answering complex logical queries over incomplete knowledge graphs (KGs) is challenging. Most previous works have focused on learning entity/relation embeddings and simulating first-order logic operators with various neural networks. However, they are bottlenecked by the inability to share world knowledge to improve logical reasoning, thus resulting in suboptimal performance. In this paper, we propose a complex logical reasoning schema over knowledge graphs upon large language models (LLMs), co…

I hear Facebook are adding Ai chat bots to all their social networks now.
Want to waste time on the computer but your friends are too busy to talk to you? Don't worry! Zuck's got you covered. Now you can talk directly to their advertising training models and they can gather profile data about you without even having to bother your friends at all!
Social networks were already about cutting you off from your friends to promote celebrity and advertising content into your face instead, and now they don't even need the celebrities. They can just fake up what they would say.

This https://arxiv.org/abs/2312.00025 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCR_…

Secure Transformer Inference Protocol
Security of model parameters and user data is critical for Transformer-based services, such as ChatGPT. While recent strides in secure two-party protocols have successfully addressed security concerns in serving Transformer models, their adoption is practically infeasible due to the prohibitive cryptographic overheads involved. Drawing insights from our hands-on experience in developing two real-world Transformer-based services, we identify the inherent efficiency bottleneck in the two-party as…

This https://arxiv.org/abs/2310.13289 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSD_…

SALMONN: Towards Generic Hearing Abilities for Large Language Models
Hearing is arguably an essential ability of artificial intelligence (AI) agents in the physical world, which refers to the perception and understanding of general auditory information consisting of at least three types of sounds: speech, audio events, and music. In this paper, we propose SALMONN, a speech audio language music open neural network, built by integrating a pre-trained text-based large language model (LLM) with speech and audio encoders into a single multimodal model. SALMONN enable…

Real-GDSR: Real-World Guided DSM Super-Resolution via Edge-Enhancing Residual Network
Daniel Panangian, Ksenia Bittner
https://arxiv.org/abs/2404.03930 htt…

Real-GDSR: Real-World Guided DSM Super-Resolution via Edge-Enhancing Residual Network
A low-resolution digital surface model (DSM) features distinctive attributes impacted by noise, sensor limitations and data acquisition conditions, which failed to be replicated using simple interpolation methods like bicubic. This causes super-resolution models trained on synthetic data does not perform effectively on real ones. Training a model on real low and high resolution DSMs pairs is also a challenge because of the lack of information. On the other hand, the existence of other imaging m…

The US and UK sign an agreement on how to test and assess risks from emerging AI models, marking the first bilateral arrangement on AI safety in the world (Madhumita Murgia/Financial Times)
https://t.co/lCMQ7i3i6k

Beyond Language Models: Byte Models are Digital World Simulators
Shangda Wu, Xu Tan, Zili Wang, Rui Wang, Xiaobing Li, Maosong Sun
https://arxiv.org/abs/2402.19155

Beyond Language Models: Byte Models are Digital World Simulators
Traditional deep learning often overlooks bytes, the basic units of the digital world, where all forms of information and operations are encoded and manipulated in binary format. Inspired by the success of next token prediction in natural language processing, we introduce bGPT, a model with next byte prediction to simulate the digital world. bGPT matches specialized models in performance across various modalities, including text, audio, and images, and offers new possibilities for predicting, s…

Doing Personal LAPS: LLM-Augmented Dialogue Construction for Personalized Multi-Session Conversational Search
Hideaki Joko, Shubham Chatterjee, Andrew Ramsay, Arjen P. de Vries, Jeff Dalton, Faegheh Hasibi
https://arxiv.org/abs/2405.03480

Doing Personal LAPS: LLM-Augmented Dialogue Construction for Personalized Multi-Session Conversational Search
The future of conversational agents will provide users with personalized information responses. However, a significant challenge in developing models is the lack of large-scale dialogue datasets that span multiple sessions and reflect real-world user preferences. Previous approaches rely on experts in a wizard-of-oz setup that is difficult to scale, particularly for personalized tasks. Our method, LAPS, addresses this by using large language models (LLMs) to guide a single human worker in gener…

Multi-modal perception for soft robotic interactions using generative models
Enrico Donato, Egidio Falotico, Thomas George Thuruthel
https://arxiv.org/abs/2404.04220

Multi-modal perception for soft robotic interactions using generative models
Perception is essential for the active interaction of physical agents with the external environment. The integration of multiple sensory modalities, such as touch and vision, enhances this perceptual process, creating a more comprehensive and robust understanding of the world. Such fusion is particularly useful for highly deformable bodies such as soft robots. Developing a compact, yet comprehensive state representation from multi-sensory inputs can pave the way for the development of complex c…

This https://arxiv.org/abs/2312.00849 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
Multimodal Large Language Models (MLLMs) have recently demonstrated impressive capabilities in multimodal understanding, reasoning, and interaction. However, existing MLLMs prevalently suffer from serious hallucination problems, generating text that is not factually grounded in associated images. The problem makes existing MLLMs untrustworthy and thus impractical in real-world (especially high-stakes) applications. To address the challenge, we present RLHF-V, which enhances MLLM trustworthiness…

This https://arxiv.org/abs/2111.10553 has been replaced.
link: https://scholar.google.com/scholar?q=a

Degree-corrected distribution-free model for community detection in weighted networks
A degree-corrected distribution-free model is proposed for weighted social networks with latent structural information. The model extends the previous distribution-free models by considering variation in node degree to fit real-world weighted networks, and it also extends the classical degree-corrected stochastic block model from un-weighted network to weighted network. We design an algorithm based on the idea of spectral clustering to fit the model. Theoretical framework on consistent estimati…

This https://arxiv.org/abs/2212.03223 has been replaced.
link: https://scholar.google.com/scholar?q=a

Financial Risk Management on a Neutral Atom Quantum Processor
Machine Learning models capable of handling the large datasets collected in the financial world can often become black boxes expensive to run. The quantum computing paradigm suggests new optimization techniques, that combined with classical algorithms, may deliver competitive, faster and more interpretable models. In this work we propose a quantum-enhanced machine learning solution for the prediction of credit rating downgrades, also known as fallen-angels forecasting in the financial risk mana…

This https://arxiv.org/abs/2309.12089 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

HiCRISP: An LLM-based Hierarchical Closed-Loop Robotic Intelligent Self-Correction Planner
The integration of Large Language Models (LLMs) into robotics has revolutionized human-robot interactions and autonomous task planning. However, these systems are often unable to self-correct during the task execution, which hinders their adaptability in dynamic real-world environments. To address this issue, we present a Hierarchical Closed-loop Robotic Intelligent Self-correction Planner (HiCRISP), an innovative framework that enables robots to correct errors within individual steps during th…

Covariance-free Multifidelity Control Variates Importance Sampling for Reliability Analysis of Rare Events
Promit Chakroborty (Dept. of Civil,Systems Engg, Johns Hopkins University), Somayajulu L. N. Dhulipala (Idaho National Laboratory), Michael D. Shields (Dept. of Civil,Systems Engg, Johns Hopkins University)
https://arxiv.or…

Covariance-free Multifidelity Control Variates Importance Sampling for Reliability Analysis of Rare Events
Multifidelity modeling has been steadily gaining attention as a tool to address the problem of exorbitant model evaluation costs that makes the estimation of failure probabilities a significant computational challenge for complex real-world problems, particularly when failure is a rare event. To implement multifidelity modeling, estimators that efficiently combine information from multiple models/sources are necessary. In past works, the variance reduction techniques of Control Variates (CV) an…

Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective
Vaisakh Shaj
https://arxiv.org/abs/2404.16078 https://

Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective
Machines that can replicate human intelligence with type 2 reasoning capabilities should be able to reason at multiple levels of spatio-temporal abstractions and scales using internal world models. Devising formalisms to develop such internal world models, which accurately reflect the causal hierarchies inherent in the dynamics of the real world, is a critical research challenge in the domains of artificial intelligence and machine learning. This thesis identifies several limitations with the p…

APrompt4EM: Augmented Prompt Tuning for Generalized Entity Matching
Yikuan Xia, Jiazun Chen, Xinchi Li, Jun Gao
https://arxiv.org/abs/2405.04820 https://…

APrompt4EM: Augmented Prompt Tuning for Generalized Entity Matching
Generalized Entity Matching (GEM), which aims at judging whether two records represented in different formats refer to the same real-world entity, is an essential task in data management. The prompt tuning paradigm for pre-trained language models (PLMs), including the recent PromptEM model, effectively addresses the challenges of low-resource GEM in practical applications, offering a robust solution when labeled data is scarce. However, existing prompt tuning models for GEM face the challenges …

APrompt4EM: Augmented Prompt Tuning for Generalized Entity Matching
Yikuan Xia, Jiazun Chen, Xinchi Li, Jun Gao
https://arxiv.org/abs/2405.04820 https://…

APrompt4EM: Augmented Prompt Tuning for Generalized Entity Matching
Generalized Entity Matching (GEM), which aims at judging whether two records represented in different formats refer to the same real-world entity, is an essential task in data management. The prompt tuning paradigm for pre-trained language models (PLMs), including the recent PromptEM model, effectively addresses the challenges of low-resource GEM in practical applications, offering a robust solution when labeled data is scarce. However, existing prompt tuning models for GEM face the challenges …

Comparative Analysis of Retrieval Systems in the Real World
Dmytro Mozolevskyi, Waseem AlShikh
https://arxiv.org/abs/2405.02048 https://

Comparative Analysis of Retrieval Systems in the Real World
This research paper presents a comprehensive analysis of integrating advanced language models with search and retrieval systems in the fields of information retrieval and natural language processing. The objective is to evaluate and compare various state-of-the-art methods based on their performance in terms of accuracy and efficiency. The analysis explores different combinations of technologies, including Azure Cognitive Search Retriever with GPT-4, Pinecone's Canopy framework, Langchain with …

This https://arxiv.org/abs/2402.03450 has been replaced.
link: https://scholar.google.com/scholar?q=a

Recommendation Fairness in Social Networks Over Time
In social recommender systems, it is crucial that the recommendation models provide equitable visibility for different demographic groups, such as gender or race. Most existing research has addressed this problem by only studying individual static snapshots of networks that typically change over time. To address this gap, we study the evolution of recommendation fairness over time and its relation to dynamic network properties. We examine three real-world dynamic networks by evaluating the fair…

"Huge generative #AI models like #ChatGPT [are] great at things I never would have expected an #LLM-based program to be good at, like writing certain kinds of computer programs, summarizing and editing text, and a whol…

Generative AI is a hammer and no one knows what is and isn’t a nail
This analogy is going to seem a bit tortured but bear with me. Imagine a world without hammers. You’re driving nails into the wall with your bare hands to hang up your paintings. You’re kicking…

This https://arxiv.org/abs/2403.01944 has been replaced.
link: https://scholar.google.com/scholar?q=a

Fourier-basis Functions to Bridge Augmentation Gap: Rethinking Frequency Augmentation in Image Classification
Computer vision models normally witness degraded performance when deployed in real-world scenarios, due to unexpected changes in inputs that were not accounted for during training. Data augmentation is commonly used to address this issue, as it aims to increase data variety and reduce the distribution gap between training and test data. However, common visual augmentations might not guarantee extensive robustness of computer vision models. In this paper, we propose Auxiliary Fourier-basis Augme…

Language Models for Code Completion: A Practical Evaluation
Maliheh Izadi, Jonathan Katzy, Tim van Dam, Marc Otten, Razvan Mihai Popescu, Arie van Deursen
https://arxiv.org/abs/2402.16197

Language Models for Code Completion: A Practical Evaluation
Transformer-based language models for automatic code completion have shown great promise so far, yet the evaluation of these models rarely uses real data. This study provides both quantitative and qualitative assessments of three public code language models when completing real-world code. We first developed an open-source IDE extension, Code4Me, for the online evaluation of the models. We collected real auto-completion usage data for over a year from more than 1200 users, resulting in over 600…

BILTS: A novel bi-invariant local trajectory-shape descriptor for rigid-body motion
Arno Verduyn, Erwin Aertbeli\"en, Glenn Maes, Joris De Schutter, Maxim Vochten
https://arxiv.org/abs/2405.04392

BILTS: A novel bi-invariant local trajectory-shape descriptor for rigid-body motion
Measuring the similarity between motions and established motion models is crucial for motion analysis, recognition, generation, and adaptation. To enhance similarity measurement across diverse contexts, invariant motion descriptors have been proposed. However, for rigid-body motion, few invariant descriptors exist that are bi-invariant, meaning invariant to both the body and world reference frames used to describe the motion. Moreover, their robustness to singularities is limited. This paper in…

Low-Res Leads the Way: Improving Generalization for Super-Resolution by Self-Supervised Learning
Haoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren, Haoze Sun, Xueyi Zou, Zhensong Zhang, Youliang Yan, Lei Zhu
https://arxiv.org/abs/2403.02601

Low-Res Leads the Way: Improving Generalization for Super-Resolution by Self-Supervised Learning
For image super-resolution (SR), bridging the gap between the performance on synthetic datasets and real-world degradation scenarios remains a challenge. This work introduces a novel "Low-Res Leads the Way" (LWay) training framework, merging Supervised Pre-training with Self-supervised Learning to enhance the adaptability of SR models to real-world images. Our approach utilizes a low-resolution (LR) reconstruction network to extract degradation embeddings from LR images, merging them with super…

This https://arxiv.org/abs/2302.03788 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

Toward a Theory of Causation for Interpreting Neural Code Models
Neural Language Models of Code, or Neural Code Models (NCMs), are rapidly progressing from research prototypes to commercial developer tools. As such, understanding the capabilities and limitations of such models is becoming critical. However, the abilities of these models are typically measured using automated metrics that often only reveal a portion of their real-world performance. While, in general, the performance of NCMs appears promising, currently much is unknown about how such models ar…

This https://arxiv.org/abs/2403.18742 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Understanding the Learning Dynamics of Alignment with Human Feedback
Aligning large language models (LLMs) with human intentions has become a critical task for safely deploying models in real-world systems. While existing alignment approaches have seen empirical success, theoretically understanding how these methods affect model behavior remains an open question. Our work provides an initial attempt to theoretically analyze the learning dynamics of human preference alignment. We formally show how the distribution of preference datasets influences the rate of mod…

Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction
Bowen Zhang, Harold Soh
https://arxiv.org/abs/2404.03868 https://

Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction
In this work, we are interested in automated methods for knowledge graph creation (KGC) from input text. Progress on large language models (LLMs) has prompted a series of recent works applying them to KGC, e.g., via zero/few-shot prompting. Despite successes on small domain-specific datasets, these models face difficulties scaling up to text common in many real-world applications. A principal issue is that in prior methods, the KG schema has to be included in the LLM prompt to generate valid tr…

This https://arxiv.org/abs/2403.01766 has been replaced.
link: https://scholar.google.com/scholar?q=a

Improving Visual Perception of a Social Robot for Controlled and In-the-wild Human-robot Interaction
Social robots often rely on visual perception to understand their users and the environment. Recent advancements in data-driven approaches for computer vision have demonstrated great potentials for applying deep-learning models to enhance a social robot's visual perception. However, the high computational demands of deep-learning methods, as opposed to the more resource-efficient shallow-learning models, bring up important questions regarding their effects on real-world interaction and user exp…

This https://arxiv.org/abs/2302.03788 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

Toward a Theory of Causation for Interpreting Neural Code Models
Neural Language Models of Code, or Neural Code Models (NCMs), are rapidly progressing from research prototypes to commercial developer tools. As such, understanding the capabilities and limitations of such models is becoming critical. However, the abilities of these models are typically measured using automated metrics that often only reveal a portion of their real-world performance. While, in general, the performance of NCMs appears promising, currently much is unknown about how such models ar…

Are We on the Right Way for Evaluating Large Vision-Language Models?
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, Feng Zhao
https://arxiv.org/abs/2403.20330

Are We on the Right Way for Evaluating Large Vision-Language Models?
Large vision-language models (LVLMs) have recently achieved rapid progress, sparking numerous studies to evaluate their multi-modal capabilities. However, we dig into current evaluation works and identify two primary issues: 1) Visual content is unnecessary for many samples. The answers can be directly inferred from the questions and options, or the world knowledge embedded in LLMs. This phenomenon is prevalent across current benchmarks. For instance, GeminiPro achieves 42.9% on the MMMU benchm…

This https://arxiv.org/abs/2404.13945 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

How do LLMs Support Deep Learning Testing? A Comprehensive Study Through the Lens of Image Mutation
Visual deep learning (VDL) systems have shown significant success in real-world applications like image recognition, object detection, and autonomous driving. To evaluate the reliability of VDL, a mainstream approach is software testing, which requires diverse and controllable mutations over image semantics. The rapid development of multi-modal large language models (MLLMs) has introduced revolutionary image mutation potentials through instruction-driven methods. Users can now freely describe d…

Compete and Compose: Learning Independent Mechanisms for Modular World Models
Anson Lei, Frederik Nolte, Bernhard Sch\"olkopf, Ingmar Posner
https://arxiv.org/abs/2404.15109 …

Compete and Compose: Learning Independent Mechanisms for Modular World Models
We present COmpetitive Mechanisms for Efficient Transfer (COMET), a modular world model which leverages reusable, independent mechanisms across different environments. COMET is trained on multiple environments with varying dynamics via a two-step process: competition and composition. This enables the model to recognise and learn transferable mechanisms. Specifically, in the competition phase, COMET is trained with a winner-takes-all gradient allocation, encouraging the emergence of independent …

This https://arxiv.org/abs/2405.01649 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
Answering complex logical queries over incomplete knowledge graphs (KGs) is challenging. Most previous works have focused on learning entity/relation embeddings and simulating first-order logic operators with various neural networks. However, they are bottlenecked by the inability to share world knowledge to improve logical reasoning, thus resulting in suboptimal performance. In this paper, we propose a complex logical reasoning schema over knowledge graphs upon large language models (LLMs), co…

This https://arxiv.org/abs/2302.03788 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

Toward a Theory of Causation for Interpreting Neural Code Models
Neural Language Models of Code, or Neural Code Models (NCMs), are rapidly progressing from research prototypes to commercial developer tools. As such, understanding the capabilities and limitations of such models is becoming critical. However, the abilities of these models are typically measured using automated metrics that often only reveal a portion of their real-world performance. While, in general, the performance of NCMs appears promising, currently much is unknown about how such models ar…

RoboDreamer: Learning Compositional World Models for Robot Imagination
Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit-Yan Yeung, Chuang Gan
https://arxiv.org/abs/2404.12377 <…

RoboDreamer: Learning Compositional World Models for Robot Imagination
Text-to-video models have demonstrated substantial potential in robotic decision-making, enabling the imagination of realistic plans of future actions as well as accurate environment simulation. However, one major issue in such models is generalization -- models are limited to synthesizing videos subject to language instructions similar to those seen at training time. This is heavily limiting in decision-making, where we seek a powerful world model to synthesize plans of unseen combinations of …

This https://arxiv.org/abs/2403.03627 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Multimodal Large Language Models to Support Real-World Fact-Checking
Multimodal large language models (MLLMs) carry the potential to support humans in processing vast amounts of information. While MLLMs are already being used as a fact-checking tool, their abilities and limitations in this regard are understudied. Here is aim to bridge this gap. In particular, we propose a framework for systematically assessing the capacity of current multimodal models to facilitate real-world fact-checking. Our methodology is evidence-free, leveraging only these models' intrins…

This https://arxiv.org/abs/2309.02561 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

Physically Grounded Vision-Language Models for Robotic Manipulation
Recent advances in vision-language models (VLMs) have led to improved performance on tasks such as visual question answering and image captioning. Consequently, these models are now well-positioned to reason about the physical world, particularly within domains such as robotic manipulation. However, current VLMs are limited in their understanding of the physical concepts (e.g., material, fragility) of common objects, which restricts their usefulness for robotic manipulation tasks that involve i…

Model-less Is the Best Model: Generating Pure Code Implementations to Replace On-Device DL Models
Mingyi Zhou, Xiang Gao, Pei Liu, John Grundy, Chunyang Chen, Xiao Chen, Li Li
https://arxiv.org/abs/2403.16479

Model-less Is the Best Model: Generating Pure Code Implementations to Replace On-Device DL Models
Recent studies show that deployed deep learning (DL) models such as those of Tensor Flow Lite (TFLite) can be easily extracted from real-world applications and devices by attackers to generate many kinds of attacks like adversarial attacks. Although securing deployed on-device DL models has gained increasing attention, no existing methods can fully prevent the aforementioned threats. Traditional software protection techniques have been widely explored, if on-device models can be implemented usi…

This https://arxiv.org/abs/2402.12147 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models
In this paper, we explore the challenges associated with establishing an end-to-end fact-checking pipeline in a real-world context, covering over 90 languages. Our real-world experimental benchmarks demonstrate that fine-tuning Transformer models specifically for fact-checking tasks, such as claim detection and veracity prediction, provide superior performance over large language models (LLMs) like GPT-4, GPT-3.5-Turbo, and Mistral-7b. However, we illustrate that LLMs excel in generative tasks …

ADM: Accelerated Diffusion Model via Estimated Priors for Robust Motion Prediction under Uncertainties
Jiahui Li, Tianle Shen, Zekai Gu, Jiawei Sun, Chengran Yuan, Yuhang Han, Shuo Sun, Marcelo H. Ang Jr
https://arxiv.org/abs/2405.00797

ADM: Accelerated Diffusion Model via Estimated Priors for Robust Motion Prediction under Uncertainties
Motion prediction is a challenging problem in autonomous driving as it demands the system to comprehend stochastic dynamics and the multi-modal nature of real-world agent interactions. Diffusion models have recently risen to prominence, and have proven particularly effective in pedestrian motion prediction tasks. However, the significant time consumption and sensitivity to noise have limited the real-time predictive capability of diffusion models. In response to these impediments, we propose a …

LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report
Justin Zhao, Timothy Wang, Wael Abid, Geoffrey Angus, Arnav Garg, Jeffery Kinnison, Alex Sherstinsky, Piero Molino, Travis Addair, Devvret Rishi
https://arxiv.org/abs/2405.00732

LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report
Low Rank Adaptation (LoRA) has emerged as one of the most widely adopted methods for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs). LoRA reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning. We aim to assess the viability of training and serving LLMs fine-tuned with LoRA in real-world applications. First, we measure the quality of LLMs fine-tuned with quantized low rank adapters across 10 base models and …